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Abstract 

A  table  of  useful  summation  formulae  are  derived,  together  with  a 
Mathematica  package  for  producing  them.  The  distance  distribution 
in  mesh  routing  networks  is  derived.  The  mean  and  variance  of  the 
distance  distribution  are  computed.  A  program  for  computing  the 
distance  distribution  of  any  mesh  is  presented. 


1  Introduction 

The  author’s  study  of  mesh  distance-distributions  was  prompted  by 
his  study  of  routing  network  simulator  convergence.  The  variance  of 
the  path-length  distribution  in  a  mesh  routing  network  effects  the  con¬ 
vergence  of  simulators  [1]  used  for  studying  routing  networks  [2].  For 
a  fine-grained  multicomputer  [3],  such  as  the  Mosaic  [4],  the  number  of 
distinct  ( src,dst )  pairs  is  too  large  1  to  simulate  every  path.  Various 
network  parameters  converge  to  their  true,  asymptotic  values  as  the 
average  distance  of  the  sampled  paths  converges  to  the  mean  distance. 

The  first  section  derives  summation  formulae  used  in  subsequent 
calculations.  A  recursive  technique  for  calculating  a  family  of  formulae 
is  derived,  and  it  is  used  to  deduce  the  needed  equations.  A  Mathe¬ 
matica  [5]  package  that  can  be  used  to  compute  any  formula  in  the 
family  is  also  derived. 

The  second  section  develops  a  recursive  technique  for  computing 
the  distance  distribution  of  any  mesh.  The  distributions  of  one-  and 
two-dimensional  meshes  are  derived  explicitly. 

1A  128  x  128  mesh  lias  268435456  ( src,dst )  pairs. 
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The  third  section  computes  the  mean  and  variance  of  the  distance 
distribution.  Calculating  the  variance  was  the  original  motivation  for 
studying  the  distribution,  but  it  was  discovered  that  the  moments  of 
the  distribution  could  be  computed  without  knowing  the  distribution. 

The  last  section  presents  a  simple  program  that  directly  computes 
the  distance  distribution  of  any  mesh.  This  program  can  be  used  to 
check  the  analytical  results  and  to  measure  properties  for  which  no 
equations  are  presented. 


2  Summation  Formulae 

This  section  derives  summation  formulae  used  in  calculations.  A 
shorthand  notation  is  defined  to  simplify  the  exposition.  Lemmas 
present  simple  facts  used  in  the  proofs.  A  theorem  is  proven  that  pro¬ 
vides  a  recursive  technique  for  computing  the  desired  formulae.  The 
summation  formulae  are  derived  as  corollaries. 

Definition  1  For  integer  h  >  l  >  0  and  n  >  0,  let  Sfh  =  J2i=i 


Lemma  1  =  S£&  -  S”i_1  for  l  >  1. 

proof:  S"  *  =  Ef=o  in  =  EU  +  EL  in  =  +  Sfr 

Lemma  2  Sq  h  =  h  -f  1 

proof:  Induction  on  h.  Sq  0  =  1.  Sq  h  =  Sq  h_1  +  1  for  h  >  1. 
Corollary  1  S®  h  —  h  —  l  +  1 

proof:  Sq  h  =  h  -(-  1  and  =  Sq  h  —  Sq  ;_1  =  h  +  1  —  l  for  l  >  1. 

Lemma  3  Sq  h=  Sf  h  if  and  only  if  n  >  0. 
proof:  0”  =  0  if  and  only  if  n  >  0. 

Theorem  1  Sq  h  =  Ef=i  for  n  >  1. 


2 


proof:  Induction  on  h.  Base  case: 

•  Sq0  =  0  for  n  >  1 

•  zE/Li  ( anything )  =  0 
Induction  step: 

ora  _  on  I  rra 

°0,h  ~  S0,h-1  +  n 


E  s”di  +  (fc  - 1)*”-1  +  ft”-1 

/=1 

h— 1  /i— 1 

E  s?d.  +  E  '•”-1 + a-1 


i=i 
h— 1 


i=l 


E  fei  + 1*"-1)  + 

l=i 

h—l 

Eon— 1  i  ora— 1 
Sl,h  +  Sh,h 


1=1 

E  s^-1 

/=! 


Corollary  2  S^h  =  ^±Ii  _  dlzll 


proof: 


S1 

so,h 


XX* 

l=i 

X>-*  + 1) 

/=i 

h 

h(h  \-\)  y.1 

i=i 

h(  h  +  1)  -  S lth 
h(h+  1) 
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-  ELi  So,7-i  forn>0 

h 

on  _  \  on— 1 

°0 ,h  ~  2_^  °l,h 

1=1 

=  E  (ss,*‘ -  s«‘,7-i) 

1=1 

h 

-  u C"-1  I  Y'  C"-1 

—  U;:>0,h  '  °0,i-l 

1=1 

Corollary  4  S|&  =  (^)2  -  (^)2 

proof: 

h 

s3  —  fts2  —  Y^  sn_1 

^o,/!  —  2-J  °0,/-l 

/=1 


Lemma  4  Sq^  = 
proof: 


4 


=  h 


'h2(h  +  1)  +  h(h  +  1)^  ^  ^I(/  -  l)2  +  /(/  -  1)' 


6 


6 


h3(h  +  1)  h2(h  +1)  1 


+ 


1 


1 


6 


-  3^+^-  gS^ 


3  / h3(h  +  1)  h2(h  +  1) 


h2(h  +  1) 


2.1  General  Sum  Formulae  in  Mathematica 

The  recursive  technique  is  simple  but  tedious;  this  suggests  automa¬ 
tion.  The  technique  will  be  adapted  for  use  with  Mathematica  [5]. 
The  required  Mathematica  input  will  be  listed,  and  a  table  of  sample 
results  will  be  given. 

The  Mathematica  package  will  use  a  refinement  of  our  sum  nota¬ 
tion.  To  adapt  our  technique  to  dynamic  programming,  a  notation  is 
introduced  for  the  coefficients  of  the  powers  of  h  in  a  series  expansion 
nf 

OI  so,h- 

Definition  2  Our  Mathematica  sum  notation  is:  s[n,l>h]=  S"^. 
The  notation  for  the  case  when  l  =  0  is:  s  [n,x]  =s  [n,0,x]  .  When 
the  upper  limit  is  implicitly  the  variable  h,  there  is  only  one  parameter: 

s  [n]  =s  [n,h]  . 


Definition  3  The  notation  for  the  coefficient  ofh1  in  s  [n]  is  c  [n,  i]  . 
That  is,  c  [n,  i]  =Coeff  icient  [s  [n]  ,  hf  ,  and  s  M=Er=oc[n.k  ]hk. 


Lemma  5  For  n  >  0,  s [n]  =  (h  +  l)s[n  —  1]  —  c[n  —  1, k]  s[k]. 

proof: 

h—  1  h  OO 

s[n]  =  -  E  so,71  =  (h+1)  S”^1  -  EE  c[n  —  1,  k]  lk 

1=0  1=0  k= 0 
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Lemma  6  The  series  expansion  of  s  [n]  is  finite:  c  [n,k]  =  0  for  k  > 
n  +  1,  so  s  [n]  =  c[n;k]/i^.  In  particular:  s[n]  =  +  0 (hn). 

proof:  Induction  on  n.  Base  case:  s[0]  =  h  +  1.  Induction  step: 

given  that  for  all  m  <  n  we  have  c[m  —  1,  m]  =  yj-  and  c[m  —  1,  k]  =  0  for 
k  >  to,  we  show  that  c[n,  n  +  l]  =  y-py  and  c[n,  k]  =  0  for  k  >  n  +  1. 


(,  +  i) 
(^) 


(1  +  c[n  -  1,  n])  s[n]  = 


( h  +  l)s[n  -  1]  -  'y  '  c[n  -  1,  k]  s[k] 

k=  0 
n  —  1 

O  +  l)s[n  -  1]  -  ^  ^  c[n  -  1,  k]  s[k] 
k= 0 

('■  +  1  -  J^cIn  -  Vk]  f^-j-p  +  o 0% 

n  \ 


h."+1 


n 


■  +  0(ftn) 


The  recurrence  relation  needed  to  compute  any  s  [n]  by  dynamic 
programming  follows  immediately  from  the  lemmas. 

Theorem  2  For  n  >  0, 

(n—  1 

(h+  l)s[n-  1]  -  J2  c[n 

k= o 

The  listing  below  illustrates  the  use  of  the  theorem  with  Mathe- 
matica,  and  the  table  gives  some  sample  results. 

c[n_,j_] :=  c[n,j]=  Coefficient [Collect [s [n] ,h] ,h~j] 
s [0] =  h+1 

s[n_]:=  s[n]=  Expand [(n/ (n+1) )( (h+1 )s  [n-1] -Sum[c [n-1 ,k]  s [k] , {k ,0 ,n-l}] )] 
TeXForm [Table [s [i] ,{i,0,10}]] 


-  l,k]s[k] 


lO 

'0,h  ~ 

1  +  h 

(i) 

h  h 2 

(2) 

!0,/i  = 

2  +  ~2 

h  h 2  h3 

(3) 

'0,h  = 

6 +  t  +  t 

h2  h 3  h4 

(4) 

[o,h  — 

t  +  y  +  t 

6 


s7 

S0,h 


-h  h3  h4  h5 

30" +  y  +  Y  +  I" 

- h 2  5  h4  h5  h6 

YY  +  Y2  +  Y  +  Y 

h  h3  h5  h6  h 7 

Y  +  Y  +  Y  +  Y 

k 2  7  h4  7  h6  h 7  h8 

U~YY  +  Y2+Y  +  Y 

-h  2  h3  7h5  2h7  h8  h9 
30"  +  ~9  15_  +  _3_+y  +  "9" 

-3  h2  h 4  7h6  3  h8  h 9  h10 

20  +  ~2  nr  +  y_  +  y  +  To’ 

5  h  h3  5  7  5/i9  h10  h11 

+  /l5-/,'+  +  + 


(5) 

(6) 

(7) 

(8) 

(9) 

(10) 

(11) 


3  Distance  Distribution 


For  a  mesh  of  radix  R  and  dimension  d,  N^(I)  denotes  the  number  of 
( src,dst )  pairs  separated  by  distance  l. 


Definition  4 


Nko  = 


(“i.  °d).  (6i.  6d) 


<  h,  o  <  bj  <  fi,  it.,  ~  b,  i  =  i 


Lemma  7  N^(0)  = 

proof:  N^(0)  =  |{(al5 ad)  |  0  <  at  <  i?}| 

Lemma  8  N^(I)  =  0  for  l  >  d(R  —  1) 
proof:  \ai  —  b{\  <  R  —  1 


Lemma  9 


N  U0 


R  1  =  0 

2(R  —  /)  1  <  /  <  -R  —  1 

0  l  >  R 
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proof:  For  1  <  /  <  R  —  1: 

Njj(I)  =  2  ■  \{{a,b)  \  0  <  a  <  b  <  R,  b  —  a  =  l}\ 

=  2  •  | {(0,  /),(!,/+  1), (i?  —  l  -  1,R—  1)}| 


Theorem  3  NdR(l)  =  E!=o  ^  “  *) 

proof: 


Nfl(0  = 


(“1.  ad).  ( 6d) 


■  E 

«=0 

l 

-  E 

«=0 

l 

-  E 


<  <  ii,  0  <  i>*  <  B,  I aj  -  bjl  = 

i=l 

1  I  /  \  a— 1 

<  (("I,-,  «d)>(6l>  >6d)]  0  <  <  ^0  <  bi  <  R,  l«d  -  bd\  =  *»  \aj  ~bj\  = 

»=0  l  '  '  J=1 

d—1 

<ai<R,0<bi<R,\ad-bd\  =  i,  ^  \aj  -  b;  \  = 

3  =  1 

^  1  / 

/  v  laj  -  bjl  =  *  -  *  }  x  \  (ad>  bd ) 

d—1 

E  |aj  -  6j 1 = ;  * 


(“1.  “d)’(6l'  6d) 


(“1  “d  })  (*1.  6d-l3 


(“J  <*d-l).(bl-  6d-l) 


l“d  -  bdl 


i= i 


(ad- 6d) 


l“d  “ 


Corollary  5  For  0  <  /  <  F 

;-i 

N|j(/)  =  2(F  -  l)!?-1  +  RNdR-\l)  +  ^  2  (R-1  +  i)  N  ^(i) 

2  =  1 

proof: 

N^(/)  =  M^W  +  E2^-')^^-*) 

i- 1 

=  fn^-1(/)  +  x:2(f-/  +  j)n|-1o-) 

j=o 

/-i 

=  M^(I)  +  2(F  -  +  £  2(F  -  /  +  j)  N^j) 

j=i 
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Corollary  6  For  l  >  R, 


Nto  =  M^(l)+  E  2(R-l  +  i)Nd£1{i) 

l  — l  — 

proof:  N dR(l)  =  RNdR~\l)  +  EiV  2 (R  ~  *)  N d£\l  -  i) 

Corollary  7 


'  R 2 

l  =  0 

AR2l  (l 

_  _L  +  JL  _ 

R  T  6 R2 

6  R?  ) 

1  <  /  <  R  -  1 

31  ,  3 l2 

2  R  '  4  R2 

_  J! _ 3_  + 

8 R3  4 R2  '  8R3  J 

R  <  l  <  2(  R  —  1) 

.  0 

/  >  2R  -  1 

proof:  For  0  <  l  <  R, 

l- 1 

N 2R(l)  =  2(R-  l)-R  +  R-2(R-l)  +  YJ2(R-l  +  i)-  2(R-i) 

i= 1 

=  ARl(R-l)  +  Al  s[l,  1,1—  1]  —  4  s[2, 1,1—  1] 

For  R  <  l  <  2(R  —  1), 

J?-i 

N|j(/)  =  E  2(J2  -/  +  *)■  2 (£-0 

=  4£(£-  l)(2R  —  l  -  1)  +  4/  s[l,  1  —  R  +  1,  R  —  l] 

—4  s[2, 1  —  R+  1, R  —  1] 


4  Moments  of  the  Distribution 

In  this  section,  the  mean  distance  and  mean-square  distance  are  com¬ 
puted  to  allow  calculation  of  the  variance.  Three  theorems  are  proven. 
The  first  theorem  states  that  the  average  distance  in  a  d-dimensional 
mesh  is  d  times  the  average  distance  in  a  one- dimensional  mesh.  The 
second  theorem  states  that  in  a  d-dimensional  mesh  the  variance  of  the 
distance  is  d  times  the  variance  of  the  distance  in  a  one- dimensional 
mesh.  The  third  theorem  states  that  the  standard  (RMS)  deviation 
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of  the  distance  divided  by  the  average  distance  for  a  d-dimensional 
mesh  is  times  the  value  for  a  one- dimensional  mesh.  Moments  of 
one-dimensional  meshes  are  computed  as  lemmas  to  provide  explicit 
formulae  for  arbitrary  meshes. 

Definition  5  (Dn)dR  =  ^E?^  '  N dR(l) 


Lemma  10  ( D°)R  =  1 

proof: 

d(R-l) 

E  Nfl(0  =  |{((«1J  <*d),  (bi,  |  0  <  <  R,  0  <  bi  <  i?}|  =  R2d 

/=0 


Lemma  11  (D1)^  =  |  (r  -  ^ j 

proof:  E;=i  L  2(1?  —  /)  =  ^  (2/2  Sj  R1  —  2  Sq 


Lemma  12 


R- 1  R- 1  R- 1  R- 1 


•••  E  E---E  (|ai-6i|  +  ...  +  |a,-6d 


aj=0  ad=0  61=0  6d=0 


proof: 


E  ia_  bi"  =  E  |{(a’b) 

a,b  l 


a  -  D  = 


b|  =  /} 


Theorem  4  (D1)^  =  §  (r  - 
proof:  (D1)^  =  d  •  (D1)^  because 


B-i  B-i  B-i  R-i 

E  •  •  •  E  dai  -  6ii  +  •  •  •  +  \a<L  -  M)  =  d  e  E  i°i  - 


ai=0  6d=0 


ai=0  =0 
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Lemma  13  (D2)^  =  \{R2  —  1) 

proof:  £a=o  Ef^a  -  b)2  =  2  R  S2^  -  2 

Lemma  14  (I?2)^  =  ^g^E2  -  +  ggg,  for  d>  1. 

proof: 


1 

tf2^ 


i  E  •  •  •  E  E k  -  k  E  k  -  bi 

a\  =0  6^=0  2=1  j= 1 

i?-l  i?-l  /  d  d 


R- 1  il-1  /  d  > 

E  •  •  •  E  Ek-  k 

ai=0  6d=0  V*=l  > 

R-l  R-l  d  d 


1 


=  -ffu  E  •  •  •  E  I E E  k  -  kk  -  k  +  E  k  - k2 


a1=°  6d=0  \!=1  jjti 


8=1 


d  /  1  i?-l  i?-l  \  /  1  i?-l  #-l 

E  E  ( £2  E  E  k  -  k  ]  (  E  E  k  - 


i?2 

8=1  j 8  \  a*=0  6*=0 


a  j  =0  b  .  =0 


1 


1?-1  J?-l 


+  E  £2  E  E  k-k2 

2=1  y  a*=0  6j=0 


Corollary  8  (L>2}^  =  § i?2  -  f  +  §  Jr 


Theorem  5  {D2)dR  -  ({D1}^  =  d-  (^(D2)^  -  ((-D1)^)  for  d  >  1 

proof: 

(<d  - 1)  (p1)^)  2  +  k^2}i)  -  (d^i)2  =  d(^2)Ed  (k1^) 2 


n 


Theorem  6 


5  Program  for  Distribution 

This  section  presents  a  simple  program  that  computes  the  mesh  dis¬ 
tance  distribution  directly  for  a  specified  radix  and  dimension.  The 
program  does  not  use  any  of  the  analysis,  so  it  provides  independent 
confirmation  of  results.  Since  it  computes  the  distribution  directly 
(brute  force),  the  program  is  very  short,  simple,  and  obviously  cor¬ 
rect.  The  program  could  be  made  more  efficient,  but  it  is  fast  enough 
for  practical  purposes. 

The  program  can  compute  the  distance  distribution  for  any  mesh, 
and  the  distribution  can  be  used  to  compute  any  desired  statistic.  The 
closed-form  expression  for  the  mesh  distance  distribution  is  tedious  to 
derive  and  use  for  dimensions  greater  than  2,  so  it  is  preferable  to 
use  the  program  for  high- dimensional  meshes.  An  example  of  the  pro¬ 
gram’s  utility  is  that  it  not  only  computes  the  mean  and  standard 
deviation  (for  which  simple  formulae  were  derived  in  the  previous  sec¬ 
tion),  but  it  also  computes  the  probability  that  a  distance  lies  within 
one  standard  deviation  of  the  mean  (a  useful  figure  for  which  no  for¬ 
mula  is  given).  The  program  listing  appears  in  Figure  1,  and  Table  1 
gives  sample  results. 
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/*  ddist.c 

*  Invoking  "ddist  R  d"  computes  the  distance  distribution 

*  for  a  radix-R  d-dimensional  mesh. 

*/ 

#include  <stdio.h> 

#include  <math.h> 

#include  <malloc.h> 

main(argc,argv)  char  **argv;  int  argc; 

{ 

int  R=atoi(argv  [1] ) ,  d=atoi(argv[2] ) ; 
int  U=(int)pow( (double)R, (double)d) ; 

int  i,j .stride, dim, dist,l,m;  double  m0=0. ,ml=0. ,m2=0. ,sd; 

double  *c= ( double* )malloc( (unsigned) ( (d*(R-l)+l)*sizeof (double) ) ) ; 

printf  ("R=,/,d,  d=*/.d,  N=*/.d\n"  ,R,d,N) ; 

for(i=0;  i<=d*(R-l);  i++)  c[i]=0.; 

for(i=0;  i<I ;  i++)  for(j=0;  j<N;  j++)  {. 

for(dist=dim=0,  stride=l;  dim<d;  dim++,  stride*=R) 
dist+=abs((i/stride)’/,R  -  (  j /stride  )"/,R) ; 
c[dist]++;  }■ 

for(i=0;  i<=d*(R-l);  i++) 

printf  (  "  count  [*/,d]  =y,g\n"  ,  i ,  c  [i]  ) , 
mO+=c[i] ,ml+=i*c[i] ,m2+=i*i*c [i]  ; 
printf  ("#paths  =  °/,g  (U*2=y,d)\n"  ,mO,]J*I) ; 

printf  ("avg(D)  =  */,g  (d*(R-l/R)/3=y,g)\n"  ,ml/=mO,d*(R-l . /R)/3  . ) ; 

printf  ("avg(D*2)=y,g  \n"  ,m2/=m0)  ; 

printf  ("std  dev=  y,g\n"  ,sd=sqrt(m2-ml*ml) )  ; 

printf  ("std  dev/avg(D)=y,g\n"  ,sd/ml)  ; 

printf  ("range  ('/.d.'/.d)  "  ,l=(int)  (ml-sd)  ,m=(int)  (ml+sd+.  5) ) ; 

for(i=l,  j=0;  i<=m;  i++)  j+=c[i]; 

printf  ("confidence  y,g\n"  ,  ((double)  j  )/m0) ; 


Figure  1:  Simple  Program  for  Computing  Distance  Distribution 
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mesh 

avg  dist 

std  dev 

std  dev/avg 

range 

confidence 

4  x  1 

1.25 

.968 

.775 

0-2 

88% 

8  x  1 

2.63 

1.90 

.724 

0-5 

91% 

16  x  1 

5.31 

3.78 

.711 

1-9 

77% 

4x4 

2.50 

1.37 

.548 

1-4 

86% 

32  x  1 

10.7 

7.55 

.708 

3-18 

67% 

64  x  1 

21.3 

15.1 

.707 

6-36 

65% 

8x8 

5.25 

2.69 

.512 

2-8 

81% 

4x4x4 

3.75 

1.68 

.447 

2-5 

76% 

128  x  1 

42.7 

30.2 

.707 

12-73 

65% 

16  x  16 

10.6 

5.34 

.503 

5-16 

72% 

8x8x8 

7.88 

3.29 

.417 

4-11 

77% 

32  x  32 

21.3 

10.7 

.501 

10-32 

70% 

64  x  64 

42.7 

21.3 

.500 

21-64 

68% 

16  x  16  x  16 

15.9 

6.54 

.411 

9-22 

70% 

128  x  128 

85.3 

42.7 

.500 

42-128 

67% 

32  x  32  x  32 

32.0 

13.1 

.409 

18-45 

70% 

Table  1:  Sample  Results  from  Program 

Execution  time  is  0 (dN2)  =  0 (dR2d).  On  a  Sun  SPARCstation 
using  cc  without  optimization,  the  32  X  32  X  32  calculation  takes 
roughly  21  hours,  the  128  X  128  calculation  takes  roughly  3.5  hours, 
and  the  smaller  meshes  take  less  than  20  minutes. 
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